3D caption display method and 3D display apparatus for implementing the same

ABSTRACT

A 3D caption display method for harmoniously merging and displaying closed caption with 3D image in a 3D display apparatus, and a 3D display apparatus suitable for implementing the method. The 3D display apparatus receives a broadcast signal containing caption data, and obtains caption control commands including a 3D parameter setting command from the caption data. Afterwards, the 3D display apparatus displays a plurality of 3D caption images according to the caption control commands.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application is a Continuation of application Ser. No. 13/716,920filed on Dec. 17, 2012, which is a Continuation of application Ser. No.12/591,848 filed on Dec. 2, 2009, which claims priorities from U.S.Provisional Applications No. 61/119,337, filed on Dec. 2, 2008, No.61/120,483, filed on Dec. 7, 2008, and No. 61/260,788, filed on Nov. 12,2009, in the United States Patent and Trademark Office, the disclosuresof which are incorporated herein in their entirety by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image display method in a displayapparatus and, more particularly, to a caption display method. Also, thepresent invention relates to a display apparatus adaptive forimplementing the image display method.

2. Description of the Related Art

Closed caption broadcasting is a broadcast service of separatelytransmitting text data not merged with a television broadcast image toallow a television receiver to selectively display the text as acaption. In NTSC analog TV broadcast, caption data is transmitted by twobytes each time as analog waveforms at line 21 of first and secondfields of a vertical blanking interval (VBI), so that a receiver candisplay the caption data on its screen. Meanwhile, according to ATSCdigital television broadcasting standard, two bytes of closed captiondata can be inserted into extension_and_user_data( ) structure of apicture header as a sort of user data, and control information forreproducing the closed caption can be defined in an Event InformationTable (EIT) of Program and System Information Protocol (PSIP) (See “ATSCDigital Television Standard Part 4—MPEG-2 Video System Characteristics(A/53, Part 4:2007), with Amendment No. 1” and “Digital Television (DTV)Closed Captioning, CEA-708-B”). The receiver can accumulate the captiondata byte pairs received through the picture header according to theCEA-708-B standard to interpret and display the closed caption accordingto the definition or specification made by the transmitting party.

The DTV closed caption (DTVCC) compliant with the CEA-708-B standardwill now be described briefly with reference to FIGS. 1 through 5.

DTVCC data is transmitted according to a data communications protocol ofparticular format. FIG. 1 shows the protocol stack of the DTVCC. Theprotocol stack includes five layers: a transport layer, a packet layer,a service layer, a coding layer, and an interpretation layer.

The transport layer represents a mechanism in which the DTVCC data isinserted into a DTV video sub-system and extracted by the receiver, andis defined in the ATSC A/53 and ISO/IEC 13818 standards. In detail, theDTVCC data is divided in a unit of two bytes and transmitted throughpicture user data of DTV stream, and a signaling descriptor istransmitted through a Program Mapping Table (PMT) and/or the EIT of thePSIP.

FIG. 2 illustrates a syntax of the closed caption data cc_data( )transmitted through the picture user data. “process_cc_data_flag” is aflag indicating whether closed caption data cc_data( ) needs to beprocessed. If the flag is set to ‘1’, the closed caption data should beparsed and its meaning has to be processed, while, if the flag is resetto ‘0’, the closed caption data may be discarded. “cc_count” fieldindicates the number of caption data constructs following this field andcan have a value from 0 through 31. Each of the caption data constructsof the cc_count carries sixteen bits of caption data through a pair of“cc_data_(—)1” and “cc_data_(—)2” fields. When “cc_valid” field is setto 1, it indicates that the two caption data bytes that follow arevalid. When reset to 0, the “cc_valid” field indicates that the twocaption data bytes are invalid. “cc_type” field denotes a type of twocaption data bytes as defined in the CEA-708-B standard. For example, ifthe “cc_type” field has a value of 10, it indicates that the two captiondata bytes are channel packet data, and if the “cc_type” field has avalue of 11, it indicates the start of new channel packet data.

Referring back to FIG. 1, prior to be encoded in the transport layer,the DTVCC data is packetized into caption channel packets according to acertain rule. The packet layer is defined by the caption channelpackets. FIG. 3 shows the structure of the caption channel packet. An nbyte caption channel packet consists of a one byte packet header and n−1byte packet data. The packet header contains a sequence number andpacket size data. The sequence number is three byte data rolling withina range of 0 to 3 in order to determine whether or not a packet has beenlost.

The DTVCC caption channel may be divided into a set of logicalsub-channels called “services.” Service data are inserted into a captionchannel data stream according to a time division multiplexing scheme.The service layer defines a header for closed caption channel servicenumbers, a service type, a service attribute, and the like. The captionchannel stream can carry six standard services and up to 57 additionalextended services, and thus allows 63 total services. FIG. 4 shows thestructure of a service block with respect to each service. A serviceblock includes a service block header and service block data. Theservice block header includes a service number and service block sizedata. If the number of services is 6 or smaller, the service blockheader has one byte, while if the number of services exceeds 6, theservice block header has two bytes.

Referring back to FIG. 1, the coding layer describes how data are codedfor the closed caption services. In other words, the coding layerdefines the assignment of numeric codes for code space control, captioncommands, and caption characters and symbols. FIG. 5 shows a code spaceorganization according to the CEA-708-B standard. The code spaceconsists of two code books each having 256 positions, and are dividedinto four code groups: CL, GL, CR, and GR. The CL group includes 32codes from 00h to 1Fh, and C0 code set (miscellaneous control codes) andC2 code set (extended miscellaneous control codes) are mapped to thisspace. The GL group includes 96 codes from 20h to 7Fh, and a G0 code set(a character set) and G2 code set (extended control code set 1) aremapped to this space. The CR group includes 32 codes from 80h to 9Fh,and C1 code set (caption control codes) and C3 code set (extendedcontrol code 2) are mapped to this space. The GR group includes 96 codesfrom A0h to FFh, and G1 code set (Latin characters) and G3 code set(future characters and icon expansion) are mapped to this space.

Particularly, the C1 code set includes window commands, pen commands,caption text commands, and synchronization commands. The window commandscreate, delete, modify, and display windows, and specify the currentcaption window for a caption service. The pen commands define penattributes and colors. The synchronization commands control the rate ofservice data interpretation.

Basic codes among the four code groups (CL, GL, CR, and GR) arecharacters, control codes, and commands of C0, C1, G0 and G1 code setsat an upper portion of the code space shown in FIG. 5. C2, C3, G2, andG3 code sets at a lower portion of the code space shown in FIG. 5 can beaccessed by using an ‘EXT1’ code (10h) of the C0 code set. That is, byprefixing the codes by the ‘EXT1’ code, extended C2, C3, G2, and G3 codesets are referenced. In other words, two bytes (i.e., ‘EXT1’+basic code)must be used in order to reference characters in the C2, C3, G2, and G3code sets.

The interpretation layer defines the DTVCC graphic user interface,namely, how caption data are encoded by using the code sets of thecoding layer and how the caption data are interpreted when decoded. Theinterpretation layer deals with a caption screen, window, pen, text, anddisplay synchronization.

According to closed caption broadcasting implemented by such a protocol,words of speakers, lyrics of songs, movie lines translation, online TVguide, an emergency broadcast, and various other text services can beprovided. Recently, as closed caption broadcasting tends to becomemandatory limitedly in terms of media access rights of the hearingimpaired or universal services, the closed caption broadcasting isexpected to expand its utilization.

Meanwhile, the advancement of television technology has reached a levelof implementing a apparatus for displaying three-dimensional (3D)images, and a full-scale commercialization of a stereoscopic type 3Dtelevision is around the corner at the time of filing the presentapplication. In the stereoscopic 3D display system, two images capturedby use of two image sensors spaced apart by about 65 millimeters likethe human eyes are transmitted as broadcast signals to a receiver. Then,the receiver allows the two images to be inputted to the left and righteyes of a user to simulate the binocular disparity producing deepperception or stereopsis.

In order to implement the closed caption in such a stereoscopic 3Dtelevision, it is necessary to reveal the closed caption on both theleft and right images. If, however, the caption is displayed at the sameposition of the left and right images in a state that all the objectswithin the left and right images are displayed with disparity tosimulate binocular disparity, the caption display will be incomplete andbreak down the 3D display to confuse the viewers and result in aheadache or nausea.

Even though the ATSC digital television standard (A/53) and itssupplemental standard, i.e., the Program and System Information Protocol(PSIP) standard (A/65C) and the DTVCC standard (CEA-708-B) describestransmission of closed caption, the specification of these standards aremerely suitable for transmitting closed caption data for commontwo-dimensional television, not applicable for the 3D television. If 3Dcaption data are to be transmitted according to these standards, thecaption data for each image of the 3D image pair should be transmittedseparately, which results in a substantial reduction of availablebandwidth by a half causing a problem of inefficient use of resources.

Thus, a method for effectively displaying closed captions compatiblewith 3D images in a receiver without degrading the efficiency ofbandwidth resources is required.

SUMMARY OF THE INVENTION

Therefore, it is an object of the present invention to provide a 3Dcaption display method for harmoniously merging and displaying closedcaption with 3D image in a 3D display apparatus.

It is another object of the present invention to provide a 3D displayapparatus suitable for implementing the closed caption display method.

According to the present invention, when the 3D display apparatusreceives image signals of first and second images for implementing 3Dimage in a stereoscopic scheme, for example, the 3D display apparatusreceives closed caption data only for a base view determined from thefirst and second images. Then, the display apparatus determines theposition of a closed caption for an extended view based on the closedcaption data for the base view and a certain 3D caption information, andformats caption graphic data for the base view and the extended view tooutput 3D closed caption on its screen. The 3D caption information mayinclude a 3D caption parameter, i.e., a horizontal disparity at theclosed caption display position or depth information of the closedcaption display position. The 3D caption parameter may be receivedtogether with the image signals, or may be received separately. However,the 3D caption parameter may be determined by the display apparatus aswell.

In more detail, according to an aspect of the 3D caption display methodfor achieving one of the objects above, the 3D display apparatusreceives a broadcast signal containing caption data, and obtains captioncontrol commands including a 3D parameter setting command from thecaption data. Afterwards, the 3D display apparatus displays a pluralityof 3D caption images according to the caption control commands.

Preferably, a plurality of caption windows can be displayed on a screenof the 3D display apparatus. The 3D parameter setting command includes awindow ID designating a caption window for displaying the 3D captionimages. The 3D parameter setting command can be received in pluralnumbers, and each of the plural 3D parameter setting commands isdirected to respective caption window.

Preferably, the 3D parameter setting command includes horizontaldisparity or depth information for displaying the 3D caption.

Preferably, the caption data includes base view caption data.

Preferably, the step of displaying the 3D caption images comprises astep of generating extended view caption data.

It is preferable that the extended view caption data is generated by useof the base view caption data and the horizontal disparity information.

When displaying the 3D caption images, the 3D display apparatuspreferably formats the base view caption data and the extended viewcaption data to display three-dimensionally.

Meanwhile, a 3D display apparatus for achieving another one of theobjects above includes a signal receiving unit, a caption decoding unit,and an output unit. The signal receiving unit receives a broadcastsignal containing caption data. The caption decoding unit obtainscaption control commands including a 3D parameter setting command fromthe caption data. The output unit displays a plurality of 3D captionimages according to the caption control commands.

In a preferred embodiment, the signal receiving unit includes ademodulator, a demultiplexer, and a video decoder. The demodulatorreceives and demodulates the broadcast signal to output a transportstream. The demultiplexer demultiplexes the transport stream to outputan encoded video signal and signaling information. The video decoderparses the encoded image signal to restore base view image signal andextended view image signal, extracts a picture header, and extracts thecaption data from the picture header.

Preferably, the caption decoding unit extracts caption text and thecaption control commands from the caption data, and obtains horizontaldisparity information.

Preferably, the output unit includes a graphic generation unit forgenerating the extended view caption data by use of the base viewcaption data and the horizontal disparity information; and a formatterthe base view caption data and the extended view caption data to displaythree-dimensionally.

According to the present invention, closed caption can naturally bemerged with 3D image and displayed according to the 3D display apparatusof the present invention. In particular, even when closed caption datais provided only a single image rather than all of a plurality of 2Dimages for implementing a 3D image, the receiver can display the 3Dcaption based on the 3D caption information.

The present invention can be applicable to any kinds of displayapparatuses that have 3D display capability and need to have closedcaption display function. In particular, the present invention can beparticularly useful for a stereoscopic display apparatus regardless offormatting scheme such as dual mode display, a time sequential display,or the like.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will become more fully understood from thedetailed description given hereinbelow and the accompanying drawings.The same reference numerals will be used throughout the drawings todesignate the same or similar elements. In the drawings:

FIG. 1 illustrates a protocol stack of a digital television closedcaption (DTVCC) compliant with CEA-708-B standard;

FIG. 2 illustrates the syntax of closed caption data cc_data( )transmitted through picture user data according to ATSC A/53 standard;

FIG. 3 shows the structure of a caption channel packet compliant withCEA-708-B standard;

FIG. 4 shows the structure of a service block compliant with CEA-708-Bstandard;

FIG. 5 shows the configuration of a code space compliant with CEA-708-Bstandard;

FIG. 6 is a schematic block diagram of a broadcasting system accordingto an exemplary embodiment of the present invention;

FIG. 7 illustrates the syntax of user data bit stream for transmittingclosed caption data in the system of FIG. 6;

FIG. 8 illustrates the syntax of the closed caption data shown in FIG.7;

FIG. 9 illustrates the syntax of SEI RBSP payload bit stream suitablefor transmitting closed caption data in an embodiment where stereoscopicimage pair is encoded according to H.265/AVC standard;

FIG. 10 illustrates the syntax of a caption service descriptor includedin Event Information Table (EIT) transmitted by the system of FIG. 6;

FIG. 11 is a schematic block diagram of a television receiver accordingto a first exemplary embodiment of the present invention;

FIG. 12 is a flow chart illustrating the process of extracting 3Dcaption information and closed caption data in the television receiverof FIG. 11;

FIG. 13 is a flow chart illustrating the process of displaying 3D closedcaption;

FIG. 14 illustrates an example of a caption bitmap for a base view;

FIG. 15 illustrates the process of acquiring a caption bitmap for anextended view from the caption bitmap for the base view;

FIG. 16 illustrates an example of the caption bitmap for an extendedview;

FIG. 17 illustrates an example of a reference point for measuring aclosed caption depth value;

FIG. 18 shows examples of screen images obtained by formatting left andright images superimposed by respective closed caption windows;

FIG. 19 shows examples of screen images obtained by formatting left andright images superimposed by respective closed caption windows;

FIG. 20 shows examples of screen images obtained by formatting left andright images superimposed by respective closed caption windows;

FIG. 21 is a schematic block diagram of an alternative embodiment of thetelevision receiver of FIG. 11;

FIG. 22 illustrates an example of a caption bitmap image formatted in aside-by-side formay in the television receiver of FIG. 21;

FIG. 23 illustrates an example of a caption bitmap image formatted in atop-down format in the television receiver of FIG. 21;

FIG. 24 is a schematic block diagram of a broadcasting system accordingto a second exemplary embodiment of the present invention;

FIG. 25 illustrates the syntax of disparity_data( ) in which the systemof FIG. 24 transmits a 3D caption parameter;

FIG. 26 illustrates the syntax of the caption service descriptorincluded in the Event Information Table (EIT) transmitted by the systemof FIG. 24;

FIG. 27 is a schematic block diagram of a television receiver accordingto a second exemplary embodiment of the present invention;

FIG. 28 is a flow chart illustrating the process of extracting closedcaption data and displaying a 3D closed caption in the televisionreceiver illustrated in FIG. 27;

FIG. 29 is a schematic block diagram of an alternative embodiment of thetelevision receiver of FIG. 27;

FIG. 30 is a schematic block diagram of a broadcasting system accordingto a third exemplary embodiment of the present invention;

FIG. 31 illustrates an example of a coding format of Set 3D parameter(S3D) command for delivering 3D caption information;

FIG. 32 illustrates the position of the Set 3D parameter command in acode space according to an exemplary embodiment of the presentinvention;

FIG. 33 is a schematic block diagram of a television receiver accordingto a third exemplary embodiment of the present invention;

FIG. 34 is a flow chart illustrating the process of extracting closedcaption information in the television receiver of FIG. 33;

FIG. 35 is a flow chart illustrating the process of displaying 3D closedcaption in the television receiver of FIG. 33;

FIG. 36 is a table summarizing a method of interpreting and resizing thehorizontal disparity value depending on broadcast images;

FIG. 37 illustrates an exemplary command sequence for displaying 3Dcaption in the television receiver of FIG. 33;

FIG. 38 is a schematic block diagram of an alternative embodiment of thetelevision receiver of FIG. 33;

FIG. 39 illustrates another example of a coding format of the Set 3Dparameter command for delivering 3D caption information;

FIG. 40 illustrates the position of the Set 3D parameter command of FIG.36 in a code space according to an exemplary embodiment of FIG. 39;

FIG. 41 illustrates an example of using the Set 3D parameter (S3D)command along with other caption control commands;

FIG. 42 illustrates a coding format of the Set 3D parameter (S3D)command suitable for delivering the 3D parameter regardless of theactivated caption window;

FIG. 43 illustrates an example of the position of the Set 3D parameter(S3D) command of FIG. 42 in the code space; and

FIG. 44 illustrates an example of using the Set 3D parameter (S3D)command of FIG. 42 along with other caption control commands in thetelevision receiver of FIG. 33.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS First Embodiment

FIG. 6 shows a broadcasting system according to an exemplary embodimentof the present invention. The broadcasting system of FIG. 6 includes abinocular camera 100, a preprocessor 102, a program encoder 104, acontroller 112, a transport multiplexer 120, a channel coding andmodulation unit 122, and a transmitter 124.

The binocular camera 100 includes two lenses and corresponding imagepick-up devices, and captures a pair of two-dimensional (2D) images fora front view. Similarly to human eyes, the two lenses and image pickupapparatus are disposed apart by 65 millimeters (mm), and thus the camera100 acquires two 2D images having horizontal parallax. In thisdescription including appended claims, of the two 2D images constitutingstereoscopic image pair, an image acquired through the left lens will bereferred to as a left image, and another image acquired by the rightlens will be referred to as a right image.

The preprocessor 102 removes noise that may be present in the left andright original images acquired by the camera 100, corrects the images,and resolves luminance imbalancing. The images may be stored in astorage unit or edited before or after the preprocessing performed bythe preprocessor 102, and thus there may be some considerable timedifference between the image capturing by the camera 100 and imagecoding performed by the program encoder 104.

In the program encoder 104, an audio encoder 106 receives audio signalsfrom a plurality of microphones (not shown in the drawing) installed atproper locations of an image capturing spot and encodes the receivedaudio signal according to a predetermined standard, e.g., AC-3 standard,to generate an audio Elementary Stream (ES).

A video encoder 108 encodes the images acquired by the camera 100according to a certain standard and compresses the encoded images byremoving temporal and spatial redundancies to generate a videoElementary Stream (ES). The video encoder 108 receives user data from auser data inserter 116 to insert into a picture header while encodingthe images. In particular, the user data includes closed caption data.In an exemplary embodiment of the present invention, the video encoder108 encodes video signals according to MPEG-2 standard of ISO/IEC13838-2, and ATSC digital television standard of A/53 Part 4. However,without being limited thereto, the video encoder 108 may encode thevideo signals according to H.264/AVC standard drafted by ISO/IECJTC1/SC29/WG11 and Joint Video Team (JVT) of ITU-T SG16 Q.6 or otherencoding schemes.

A packetizer 110 receives the audio ES and the video ES from the audioencoder 106 and the video encoder 108, respectively, and packetizes therespective streams to generate a Packetized Elementary Stream (PES).

The controller 112 includes a caption data generator 114, a user datainserter 116, and a PSI/PSIP generator 118. The controller 112 controlsgeneral operation of the system, and generates user data, ProgramSpecification Information (PSI), and Program and System InformationProtocol (PSIP) data required for encoding the video signals.

The caption data generator 114 receives time-stamped caption datathrough a network or by recording medium from an external workstation,packetizes the received data into caption channel packets shown in FIG.3. In transmitting the stereoscopic 3D image pair, one of the two imagesmay be designated as a base view and the other one may be designated asan extended view. Preferably, caption window position information isgenerated with respect to the base view. In addition, the caption datagenerator 114 provides caption service information indicating types andattributes of caption services to the PSI/PSIP generator 118. In thepresent exemplary embodiment, the caption service information includes3D caption information such as information indicating whether or not theclosed caption service is three dimensional, parameters for supporting3D caption (referred to as ‘3D caption parameter’, hereinbelow), and thelike. The 3D caption parameter may be depth data indicating aforward/backward position of a caption in a 3D image or a horizontaldisparity indicating the difference between caption display position inthe extended view and that in the base view.

The user data inserter 116 provides the video encoder 108 with extensiondata and user data to be inserted at a sequence level, a GOP level, or apicture level during the encoding process. In particular, the user datainserter 116 provides the video encoder 108 with caption channel packetsfrom the caption data generator 114 as a sort of user data, so that thevideo encoder 108 encodes the video signal after including the captionchannel packets in the picture header in a unit of two bytes.

The PSI/PSIP generator 118 generates PSIP data. A Program Mapping Table(PMT) or an Event Information Table (EIT) in the PSIP includes a captionservice descriptor for describing caption service information. Inparticular, according to the present exemplary embodiment, the captionservice descriptor includes the 3D caption information for supportingthe 3D caption.

The transport multiplexer 120 multiplexes PES's and the PSI/PSIPinformation and adds a header to generate a Transport Stream (TS). Whilethe system of FIG. 1 transmits a single channel of terrestrial broadcastsignal, an alternative system which transmits multiple broadcast signalsvia, for example, a cable network or a satellite network mayadditionally include a transmission multiplexer for multiplexing thebroadcast signals to generate a Program Stream. The channel coding andmodulation unit 122 performs error-correction coding with respect to theTS so that a receiver can detect and correct an error that may be causedby noise in a transmission channel. In addition, the channel coding andmodulation unit 122 modulates the error-correction-coded TS according toa modulation scheme, e.g., 8-VSB modulation scheme, adopted by thesystem. The transmitter 124 transmits the modulated broadcast signalaccording to channel characteristics, e.g., via an antenna.

The structure of signaling data delivering closed caption informationwill now be described in detail with reference to FIGS. 7 through 10.

According to the MPEG-2 standard of ISO/IEC 13838-2 and ATSC digitaltelevision standard of A/53 Part 4, extension_and_user_data( ) structurefor defining extension data or user data may be inserted to a header ata video sequence, GOP, or picture level. Preferably, the closed captiondata are inserted as picture user data at the picture level, namely, inextension_and_user_data( ) that can be disposed following a pictureheader in the video ES.

A first table of FIG. 7 shows the syntax of the picture extension anduser data. A numeral ‘2’ in the parenthesis of syntax title‘extension_and_user_data(2)’ indicates that this syntax is at thepicture level. As can be seen in the execution condition of ‘do while’sentence, if bits appearing next within the picture header areextension_start_code or user_data_start_code, the following bit streamis regarded as the extension data or user data. Specifically, the bitstream following the extension_start_code is regarded as asextension_data(2), and the bit stream following the user_data_start_codeis regarded as user_data( ).

As shown in a second table of FIG. 7, picture user data (user_data( )includes “user_data_start_code” and “user_data_identifier” fields, whichare followed by user_structure( ). The value of “user_data_start_code”is set to ‘0x0000 01B2’ according to the ISO/IEC 13818-2 standard. The“user_data_identifier” field, which is a 32-bit code indicating thesyntax and meaning of user_structure( ), is set to a value of“format_identifier” as defined in the ISO/IEC 13838-1 standard. In thecase of ATSC_user_data( ) as in the present invention,“user_data_identifier” is set to a value of ‘0x4741 3934’.

The user_structure( ) which is a variable length data structure definedby the value of “user_data_identifier” field, includes“user_data_type_code” field and user_data_type_structure( ) as shown ina third table of FIG. 7. “user_data_type_code” is a 8-bit valueindicating a type of ATSC user data. When the “user_data_type_code” hasa value of ‘0x03’, it indicates closed caption data.

FIG. 8 illustrates the syntax of the closed caption data, cc_data( ).

“process_cc_data_flag” is a flag indicating whether or not the closedcaption data cc_data( ) has to be processed. When the“process_cc_data_flag” is set to 1, the closed caption data has to beparsed and processed, whereas, when the “process_cc_data_flag” is set to0, it indicates that the closed caption data can be discarded. The“cc_count” field indicates the number of caption data constructsfollowing this field, and can have a value 0 through 31. The specificvalue of the “cc_count” field is set according to a frame rate and acoded picture data structure such that a fixed bandwidth of 9600 bps ismaintained for the closed caption payload data.

For each of the cc_count number of caption data constructs, sixteen bitsof closed caption data are carried in the byte pair of “cc_data_(—)1”and “cc_data_(—)2” fields. When set to 1, “cc_valid” field indicatesthat the two caption data bytes that follow are valid. When reset to 0,“cc_valid” field indicates that the two caption data bytes that followare invalid. “cc_type” field denotes a type of two closed caption databytes as defined in the CEA-708-B standard. For example, if the“cc_type” field has a value of 10, it indicates that the two closedcaption data bytes are channel packet data, and if the “cc_type” fieldhas a value of 11, it indicates the start of a new channel packet.

In this manner, according to an exemplary embodiment of the presentinvention, two 2D images constituting the stereoscopic image pair areencoded according to the MPEG-2 standard, and the closed caption data isinserted as user data in the picture header of the video ES. In thiscase, however, the images may be encoded according to another encodingscheme, for example, the H.265/AVC standard drafted by the ISO/IECJTC1/SC29/WG11 and Joint Video Team (JVT) of ITU-T SG16. In case thatthe images are encoded according to the H.264/AVC standard, closedcaption data may be included in Supplemental Enhancement Information(SEI).

FIG. 9 illustrates the syntax of an SEI RBSP payload bit stream suitablefor transmitting closed caption data in an alternative embodiment wherethe stereoscopic image pairs are encoded according to the H.265/AVCstandard.

In the syntax shown in FIG. 9, “itu_t_t35_country_code” is a 8-bitcountry code specified in ITU-T Rec. T35, Annex A, and has a value of‘0x61’ in case of Korea. The “itu_t_t35_provider_code” is a 16-bit codehaving a value of ‘0x0031’. “user_identifier” is a 32-bit code, and canindicate that the syntax structure of user_structure( ) follows thedefinition in ATSC A/53 by a value of ‘0x4741 3934’. In such a case, theuser_structure( ) may be used in the same manner as that defined in theATSC digital television standard, namely, an A/53 Part 4 Section 6.2.3.Accordingly, closed caption data can be specified by the user datasyntax shown in the third table of FIG. 7 and the closed caption syntaxof FIG. 8.

Meanwhile, caption service information describing a type and attributesof a caption service carried by picture user data is transmitted to thereceiver separately from caption data. In the present exemplaryembodiment, the caption service information is generated according tothe ATSC A/65 standard, included in a caption service descriptor, andtransmitted through the Event Information Table (EIT) and/or the ProgramMapping Table (PMT) of MPEG-2 Transport Stream.

FIG. 10 illustrates the syntax of bit stream of the caption servicedescriptor included in the EIT according to the present exemplaryembodiment. In general, the caption service descriptor is a descriptorfor signaling closed caption information, and is used for transmittinginformation such as the type of the closed caption (e.g., NTSC line-21closed caption service or digital TV closed caption service), a captionlanguage, and the like. As mentioned above, according to the presentexemplary embodiment, the caption service descriptor includes 3D captioninformation in addition to these information. The 3D caption informationincludes information as to whether or not the closed caption service isthree dimensional, and a 3D caption parameter for supporting the 3Dcaption. The 3D caption parameter may be a caption depth value or ahorizontal disparity value between the caption display position for thebase view and that for the extended view.

“descriptor_tag” is an 8-bit field that identifies a type of adescriptor, and has a value of ‘0x86’ in case of the caption servicedescriptor. “descriptor_length” field is a count of the number of bytesfollowing this field within the descriptor. “number_of_services” fieldis a 5-bit integer in a range 1 through 16 indicating the number ofclosed caption services present in the EIT.

Each iteration in ‘for’ loop defines a single closed caption servicepresent as a single sub-stream within a closed caption stream of 9600bps. Each iteration provides the service language, attribute, andrelevant caption service number. “language” field is a 3-byte languagecode indicating the language associated with a closed caption service,and has a value specified by ISO 639.2/B standard. When “digital_cc”field is clear, the “language” field has no meaning. When set to 1, the“digital_cc” field indicates that a digital TV closed caption service ispresent. When the “digital_cc” field is clear, this field indicates thatthe NTSC line-21 closed caption service is present. “line21_field” isdeprecated and may be set to 1 or clear. “caption_service_number” fieldis a 6-bit integer in the range 0 through 63 identifying a captionservice number within the closed caption stream associated with thelanguage and attributes defined in a current iteration of the ‘for’loop. This field is defined only when the “digital_cc” field indicatesthe digital TV closed caption service. “easy_reader” field is a logicflag which indicates, when set to 1, that the closed caption servicecontains text tailored to the needs of a beginner. If this field isclear, it indicates that the closed caption service is not tailored. The“easy_reader” field has no meaning when the “digital_cc” field is clear.“wide_aspect_ratio” field is a logic flag. The “wide_aspect_ratio” fieldindicates, when set to 1, that the closed caption service is formattedfor displays with aspect ratio of 16:9. When the “wide_aspect_ratio”field is clear, the closed caption service is formatted for displayswith aspect ratio of 4:3. This field has no meaning if the “digital_cc”field is clear.

“cc_(—)3d_type” field indicates the presence of the 3D caption parameterfor a 3D stereoscopic display and a type of the 3D caption parameter.The “cc_(—)3d_type” value of ‘00’ indicates that the data bits followingthis field must be discarded regardless of the 3D caption parameter,i.e., the “cc_disparity” field or the “cc_depth” field. The“cc_(—)3d_type” value of ‘01’ indicates that the “cc_disparity” fieldshould be parsed and processed, and the “cc_disparity” field isexpressed in units of pixel numbers. The “cc_(—)3d_type” value of ‘10’indicates that the “cc_disparity” field should be parsed and processed,and the “cc_disparity” field is expressed in a ratio (%) with respect tothe width of the display. The “cc_(—)3d_type” value of ‘11’ indicatesthat the “cc_depth” field should be parsed and processed.

“base_view” field defines a base view which serves as a reference of thestereoscopic image pair, i.e., left and right images. In the presentexemplary embodiment, the left view is the base view if the “base_view”field is 0, and the right view is the base view if the “base_view” fieldis 1.

The “cc_disparity” field, which is a horizontal disparity value betweenthe base view and an extended view at a caption display position whenthe caption is displayed on a stereoscopic display, denotes the captiondisplay position for the extended view with respect to the captiondisplay position for the base view. Thus, the receiver can obtain theclosed caption display position for the extended view by shifting theclosed caption position for the base view by the horizontal disparity.As mentioned above, the data format of the “cc_disparity” field may beindicated by the “cc_(—)3d_type” field. In the above description, thehorizontal disparity value denoted by the “cc_disparity” field can beexpressed by the number of pixels or a ratio (%) with respect to thedisplay width. Alternatively, however, the “cc_disparity” may beexpressed by physical length (e.g., centimeters or inches) as well. Whenthe “cc_disparity” field is intended to be expressed by the physicallength, the “cc_(—)3d_type” field may be defined to substitute eitherthe pixel number or the ratio to the display width, or a certain fieldmay be newly defined besides the “cc_(—)3d_type” field.

The “cc_depth” field denotes the depth of the caption in thestereoscopic 3D image. When the 3D video data includes depthinformation, the depth data for the caption can be provided to thereceiver instead of the horizontal disparity (cc_disparity), so that thereceiver can determine the caption display position in the 3D images byapplying the depth data. Preferably, the size and scale of the depth maybe equal to depth information of the video data so that the receiver caninterpret it in the same manner. Even though the depth value for thevideo data may have a negative or positive value, a depth map which mapsthe depth value of a value of [0,255] may be constructed. Preferably,the caption depth is expressed in this manner. Meanwhile, inconstructing the depth map, the depth value may be mapped into a valueof [0,255] in a linear scale. Alternatively, however, the depth valuemay be mapped, for example, in a logarithmic scale in consideration ofperception characteristics of human beings.

FIG. 11 is a schematic block diagram of a television receiver forreceiving the broadcast signal transmitted by the system of FIG. 6. Thetelevision receiver according to the present exemplary embodiment issuitable for receiving the terrestrial broadcast signal to reproduceimages.

A tuner 200 selects a broadcast signal of a certain channel selected bythe user among a plurality of broadcast signals inputted via an antennanot shown in the drawing. A demodulation and channel decoding unit 202demodulates the broadcast signal from the tuner 200 and performserror-correction-decoding with respect to the demodulated signal tooutput the Transport Stream (TS). A transport demultiplexer 204demultiplexes the TS to separate a video ES and an audio ES and extractPSI/PSIP information.

An audio decoder 208 decodes the audio ES to output a digital audio bitstream. The audio bit stream is converted into an analog audio signal bya digital-to-analog converter (not shown), amplified by an amplifier(not shown), and then outputted via a speaker (not shown). A videodecoder 210 parses the video ES to extract extension and user data andan MPEG-2 video bit stream. The video decoder 210 extracts closedcaption data cc_data( ) from the extension and user data to output to acaption decoder 216. The video decoder 210 decodes the MPEG-2 video bitstream to output left and right video signals for implementing astereoscopic 3D image pair. The decoding process by the audio decoder208 and the video decoder 210 may be performed based on a packet ID(PID) detected by a PSI/PSIP processor 214.

The PSI/PSIP processor 214 receives and parses the PSI/PSIP informationfrom the transport demultiplexer 204 to store in a memory not shown inthe drawing or a register, so that the broadcast images are reproducedbased on the stored information. The EIT of the PSIP received by thereceiver includes the caption service descriptor which contains thecaption service information describing the type and attributes of acaption service. The PSI/PSIP processor 214 provides the caption serviceinformation to a caption decoder 216, so that the caption decoder 216can decode caption data based on the caption service information. Inparticular, according to the present exemplary embodiment, the captionservice information received through the caption service descriptorincludes the 3D caption information, namely, whether or not the closedcaption service is three dimensional and a 3D caption parameter forimplementing a 3D caption. Alternatively, however, the 3D captioninformation may be provided to a graphic engine 218 instead of thecaption decoder 216, or both the caption decoder 216 and the graphicengine 218.

The caption decoder 216 receives the closed caption data cc_data( ) fromthe video decoder 210 and accumulates the closed caption data pairscarried by the “cc_data_(—)1” and “cc_data_(—)2” fields in a unit ofpackets. In addition, the caption decoder 216 depacketizes the captionpacket data based on the caption service information from the PSI/PSIPprocessor 214 to restore service block data with respect to eachservice. Afterwards, the caption decoder 216 decodes and interprets theservice block data. Meanwhile, the caption decoder 216 receives thecaption service information from the PSI/PSIP processor 214 and detectsbase view information as well as the 3D caption parameter, i.e., thehorizontal disparity value or the depth value from the 3D captioninformation. The caption decoder 216 generates caption displayinformation based on the interpreted service block data and the 3Dcaption parameter to provide to the graphic engine 218.

The graphic engine 218 receives the caption display information from thecaption decoder 216 and generates caption bitmaps for the base view andthe extended view in OSD signal form based on the received captiondisplay information. Here, the base view may be the left image and theextended view may be the right image. First, the graphic engine 218constructs the caption bitmap for the base view using the base viewinformation determined by the “base_view” field and the extracted closedcaption data, and outputs the caption bitmap for the left image in OSDsignal form. Subsequently, the graphic engine 218 constructs the captionbitmap for the extended view using the 3D caption parameter and outputsthe caption bitmap for the right image in OSD signal form. When the 3Dcaption parameter is the horizontal disparity value, the caption bitmapfor the extended view can be configured by shifting the caption bitmapfor the base view by the horizontal disparity. Meanwhile, if the 3Dcaption parameter is the depth value, caption data corresponding to thebase view is displayed on the display by use of the caption depth value,which is especially useful when depth image is additionally provided inthe broadcast signal.

The first mixer 220 mixes the left image caption bitmap from the graphicengine 218 to the left image signal from the video decoder 210. Thesecond mixer 222 mixes the right image caption bitmap from the graphicengine 218 to the right image signal from the video decoder 210. Theformatter 224 formats the left and right images superimposed withrespective caption bitmaps while comparing a frame time of the signals,such that temporally coincident left and right images can be displayedas a pair on a display panel 232. The formatted 3D image signal isreconstructed into a 3D image and on the display panel 226.

Meanwhile, the image synthesis of the first and second mixers 220 and222 may be selectively performed in response to a user command. Namely,the user may manipulate, for example, a remote controller (not shown) tochoose whether the closed caption window is to be displayed or not andwhat closed caption window among a plurality of closed caption windowsis to be displayed. When such a manipulation command is applied, acontroller (not shown) of the receiver may store the command in thememory and control the display of the closed caption windows withreference to the stored command.

The process of extracting closed caption information and displaying the3D closed caption in the television receiver of FIG. 11 will now bedescribed in detail with reference to FIGS. 12 and 13.

FIG. 12 illustrates the process of extracting the closed captioninformation in the television receiver of FIG. 11;

First, the PSI/PSIP processor 214 parses the PSIP to extract a MasterGuide Table (MGT) of which “PID” is ‘0x1FFB’ and “table_ID” is ‘0xC7’based on a packet ID (PID) and a table ID, and obtains PID informationfor the Event Information Table (EIT) from the MGT. Next, the PSI/PSIPprocessor 214 reads a TP having the acquired PID to extract an EIT ofwhich “table_ID” is ‘0xC7’. And then, the PSI/PSIP processor 214 parsesthe EIT and detects a caption service descriptor shown in FIG. 10 foreach event in the EIT (step 250).

In step 252, the PSI/PSIP processor 214 parses the caption servicedescriptor to extract the caption service information (step 252). Thecaption service information received through the caption servicedescriptor includes 3D caption information, namely, whether or not theclosed caption service is three dimensional and a 3D caption parameterfor implementing the 3D caption. Thereafter, the PSI/PSIP processor 214stores the caption service information in the memory or in the register,so that the caption decoder 216 can decode the caption data based on thecaption service information and the graphic engine 218 can generate thecaption images based on the 3D caption information.

The process of storing the 3D caption information is as follows. Here,the “cc_(—)3d_type” field value determines whether to store the“cc_disparity” or “cc_depth” field value. If the “cc_(—)3d_type” fieldhas a value of ‘01’ or ‘10’ (step 254), the PSI/PSIP processor 214stores the horizontal disparity “cc_disparity” value as the 3D captionparameter (step 256). If the “cc_(—)3d_type” field has a value of ‘11’in step 254, the PSI/PSIP processor 214 stores the caption depth“cc_depth” value as the 3D caption parameter (step 258). On the otherhand, if the “cc_(—)3d_type” field has a value of ‘00’, the PSI/PSIPprocessor 214 regards the caption service as a 2D caption service, anddiscards the “cc_(—)3d_type” field and the subsequent data by regardingthem as meaningless reserved bits (step 260).

In step 262, the video decoder 210 parses the video ES to outputextension and user data and the MPEG-2 video bit stream and decodes theMPEG-2 video bit stream. In the process of parsing the video ES by thevideo decoder 210, the closed caption data cc_data( ) included in thepicture header or messages of the video ES is extracted and provided tothe caption decoder 216. Namely, the video decoder 210 detectsATSC_user_data( ) having “user_data_identifier” value of ‘0x4741 3934’from the picture user data user_data( ) of the video ES, and extractsuser_data_type_structure( ) having “user_data_type_code” of ‘0x03’ asthe closed caption data cc_data( ). Accumulating the caption data bytepairs, cc_data1 and cc_data2, received through the cc_data( ) in packetunits, the caption decoder 216 decodes and interprets the caption datapackets based on the caption service information from the PSI/PSIPprocessor 214 (step 264).

FIG. 13 illustrates the process of displaying the 3D closed caption.

In step 280, the graphic engine 218 constructs the caption bitmap forthe base view by using the decoded caption text data and base viewinformation determined by the “base_view” field. FIG. 14 shows anexample of the caption bitmap for the base view, e.g., the left image.Afterwards, the graphic engine 218 constructs the caption bitmap for theextended view by using the 3D caption parameter (step 282).

In more detail, if the 3D caption parameter stored in steps 254 to 260is the horizontal disparity value, the caption bitmap for the extendedview can be constructed by shifting the caption bitmap for the base viewby the horizontal disparity, as shown in FIG. 15. Here, if the“cc_disparity” field included in the caption service descriptor has avalue denoting a number of pixels or an actual physical length, thecaption bitmap including a caption window for the extended view can beobtained by simply shifting the caption window for the base view by thefield value. Meanwhile, if the “cc_displarity” field contained in thecaption service descriptor has a value denoting a ratio with respect tothe display width (i.e., if the “cc_(—)3d_type” field has a value of‘10’), rather than the number of pixels of the actual physical length,the horizontal disparity value in units of actual pixels can becalculated by Equation 1 below, where it is assumed that the“cc_disparity” is expressed in a resolution of N bits:

$\begin{matrix}{{{Horizontal}\mspace{14mu}{Disparity}} = \frac{{cc\_ disparity} \times {display\_ width}}{2^{N - 1}}} & \lbrack {{Equation}\mspace{14mu} 1} \rbrack\end{matrix}$

By simply shifting the caption window for the base view by thehorizontal disparity, the caption bitmap including the caption windowfor the extended view can be obtained. FIG. 16 illustrates an example ofthe caption bitmap for the extended view, e.g., the right image.

On the other hand, in case that the stored 3D caption parameter is acaption depth value, caption data corresponding to the base view isdisplayed on the display panel by using the caption depth value. Here,if the depth value obtained from the “cc_depth” field is mapped in alogarithmic scale or any other nonlinear scale rather than in a linearscale in a range of a [0,255], the depth value is inversely mapped in acorresponding manner. In this case, the depth value may be furtheradjusted by use of another information such as camera parameters or thelike.

FIG. 17 illustrates an example of a reference point for measuring theclosed caption depth value. As shown in FIG. 17, an image depth plane292 on which image focusing is made when the image is captured and thefixation point is positioned is generally located behind the displayplane 290 of the display, and may be located in front of the displayplane 290 under some circumstances (294). A depth plane 296 on which aclosed caption is displayed is preferably positioned in front of theimage depth planes 292 and 294. Here, the depth value of the closedcaption depth plane 296 is measured from the display plane 290 and has apositive value in the direction of the viewer and a negative value in abackward direction of the display plane 290.

Referring back to FIG. 13, the captions for the two views are combinedwith the image signals, which are then formatted according to anappropriate stereoscopic display scheme in step 284. The formatted videosignal is outputted through the display panel 226, and the 3D closedcaption superimposed on the 3D broadcast image is outputted on thescreen (step 286). FIGS. 18 to 20 show examples of screen imagesobtained by synthesizing the left and right images superimposed withrespective closed caption windows. Specifically, FIG. 18 shows anexample in which the left and right images are horizontally interleaved,so that the left and right images are alternately displayed line by linein the horizontal direction. FIG. 19 shows an example in which the leftand right images are vertically interleaved, so that the left and rightimages are alternately displayed line by line in the vertical direction.FIG. 20 shows an example of formatting the left and right images to havea checkerboard pattern. The displayed left and right images are inputtedto the left and right eyes of the viewer, respectively, throughpolarization glasses.

The examples of FIGS. 18 to 20 are suitable for a receiver systemutilizing polarization glasses. Alternatively, however, the presentinvention may be applicable to a receiver system utilizing shutteredglasses as well. In the receiver system utilizing shuttered glasses, theleft and right images superimposed with respective closed captionwindows shown in FIGS. 14 and 16 can be alternately displayed on thedisplay panel. And, the shuttered glasses synchronized with switching ofthe displayed images allow the left and right images to be transmittedto respective one of the left and right eyes of the viewer.

FIG. 21 is a schematic block diagram of an alternative embodiment of thetelevision receiver of FIG. 11. In the present exemplary embodiment,left and right broadcast image signals are formatted by one formatter,and the left and right closed caption OSD signals are formatted byanother formatter, and then the formatted signals are mixed. A firstformatter 224A formats the left and right image signals from the videodecoder 210 according to a stereoscopic output scheme. A secondformatter 224B formats the left and right image caption OSD signals fromthe graphic engine 218 according to the stereoscopic output scheme. Amixer 220A synthesizes output signals of the first and second formatters224A and 224B, so that the formatted image signals are displayed on thedisplay panel 226.

Assuming that the caption bitmaps for the left and right images are thesame as those shown in FIGS. 14 and 16, respectively, the secondformatter 224B outputs a caption bitmap image formatted in aside-by-side format shown in FIG. 22 or a top-down format shown in FIG.23. Accordingly, the stereoscopic 3D image displayed on the displaypanel 226 after being mixed by the mixer 220A may have a form similar tothat shown in FIG. 18 or FIG. 19. The other features of the televisionreceiver of FIG. 21 are similar to those of the embodiment of FIG. 11,and thus detailed description thereof will be omitted.

Second Embodiment

In the exemplary embodiments described above, the 3D caption informationwhich facilitates the implementation of the 3D caption by use of closedcaption data for a single view is delivered to the receiver through thecaption service descriptor of the EIT. Alternatively, however, the 3Dcaption information may be delivered in a different manner, for example,as picture user data in the video ES. FIG. 24 shows an example of abroadcasting system for implementing such a transmission scheme.

In the system of FIG. 24, a video encoder 308 receives user data from auser data inserter 316 to insert into a picture header while encodingvideo signals. In particular, the user data includes closed caption dataand 3D caption information. Both the closed caption data and the 3Dcaption information are disposed within the ATSC_user_data( ) of thepicture user data, and may be discriminated by “user_data_type_code”field as described above.

A caption data generator 314 receives time-stamped caption data througha network or by a recording medium from an external workstation, andgenerates a caption channel packet shown in FIG. 3. The caption datagenerator 314 provides the caption channel packet and the 3D captioninformation to the user data inserter 316, so that the user datainserter 316 inserts the information into the picture user data.Meanwhile, the caption data generator 314 provides caption serviceinformation indicating the type and attributes of a caption service to aPSI/PSIP generator 318 so that the PSI/PSIP generator 318 can generatean EIT.

The user data inserter 316 provides the video encoder 308 with extensiondata and user data to be inserted at a sequence level, a GOP level, or apicture level during the encoding process. In particular, the user datainserter 316 provides video encoder 308 with closed caption data and 3Dcaption information as a sort of user data, so that the video encoder308 encodes the video signal after including the closed caption data andthe 3D caption information in a picture header.

In the present exemplary embodiment, the syntax of the closed captiondata provided by the user data inserter 316 and inserted into thepicture header by the video encoder 308 has the same format as thatshown in FIGS. 7 and 8. In the present exemplary embodiment, the closedcaption data is transmitted only for one view, i.e., a base view, chosenfrom two or more views for implementing a stereoscopic 3D image. The 3Dcaption information includes information as to whether or not the closedcaption service is three dimensional, base view information, and a 3Dcaption parameter, and is used for the receiver to implement the 3Dcaption based on the closed caption data provided only for the baseview.

FIG. 25 illustrates the syntax of disparity_data( ) in which the systemof FIG. 24 transmits the 3D caption parameter.

As described above with reference to FIG. 7, the picture user datauser_data( ) includes the “user_data_start_code” and“user_data_identifier” fields, which are followed by the user_structure(). The value of the “user_data_start_code” is set to ‘0x0000 01B2’according to the ISO/IEC 13838-2 standard. The “user_data_identifier”field, which is a 32-bit code indicating the syntax and meaning of theuser_structure( ) is set to a value of “format_identifier” as defined inthe ISO/IEC 13838-1 standard. In the case of ATSC_user_data( ) accordingto the present invention, the “user_data_identifier” field is set to avalue of ‘0x4741 3934’.

The user_structure( ) is a variable length data structure defined by thevalue of “user_data_identifier”, and includes the “user_data_type_code”field and user_data_type_structure( ) as shown in a second table of FIG.25. The “user_data_type_code” is a 8-bit value indicating a type of ATSCuser data. When the user_data_type_code” field has a value of ‘0x03’, itindicates closed caption data, whereas when the user_data_type_code”field has a value of ‘0x04’, it indicates 3D closed caption parameterdata.

As shown in a third table of FIG. 25, the 3D closed caption parameterdata includes 3D caption information disparity_data( ). A fourth tableof FIG. 25 shows the syntax of the 3D caption informationdisparity_data( ). Since the meaning of each field is the same as thatdescribed above in relation to the caption service descriptorillustrated in FIG. 10, detailed description thereof will be omitted.

Referring back to FIG. 24, a PSI/PSIP generator 308 generates PSI/PSIPdata. The PSIP includes an Event Information Table (EIT), which includesa caption service descriptor for describing caption service information.Contrary to the embodiment of FIG. 7, in the present exemplaryembodiment, the caption service descriptor may follow a general formatcompliant with the A/65 standard as shown in FIG. 26, rather thanincluding 3D caption information for supporting the 3D caption.

Since the other features of the system of FIG. 24 are similar to thoseof the exemplary embodiment shown in FIG. 7, detailed descriptionthereof will be omitted.

FIG. 27 illustrates an example of a television receiver suitable forreceiving the broadcast signal transmitted by the system of FIG. 24.

The tuner 200 selects a broadcast signal of a certain channel selectedby the user among a plurality of broadcast signals inputted through theantenna. The demodulation and channel decoding unit 202 demodulates thebroadcast signal from the tuner 200, performs error-correction-decodingwith respect to the demodulated signal, and outputs the Transport Stream(TS). The transport demutiplexer 204 demultiplexes the TS to separatethe video ES and the audio ES, and extracts the PSI/PSIP information.

The audio decoder 208 decodes the audio ES to output a digital audio bitstream. The audio bit stream is converted into an analog audio signal bythe digital-to-analog converter, amplified by the amplifier, and thenoutputted through the speaker. A video decoder 410 parses the video ESto extract extension and user data and an MPEG-2 video bit stream. Thevideo decoder 410 extracts closed caption data cc_data( ) and the 3Dcaption information cc_disparity( ) from the extension and user data tooutput to the caption decoder 216. The video decoder 410 decodes theMPEG-2 video bit stream to output left and right video signals forimplementing a stereoscopic 3D image. The decoding process performed bythe audio decoder 208 and the video decoder 410 may be performed basedon a packet ID (PID) detected by the PSI/PSIP processor 214.

The PSI/PSIP processor 414 receives and parses the PSI/PSIP informationfrom the transport demultiplexer 204 to store in the memory or theregister, so that the broadcast images are reproduced based on thestored information. The EIT of the PSIP includes caption servicedescriptor which contains caption service information describing thetype and attributes of the caption service. The PSI/PSIP processor 214provides the caption service information to a caption decoder 416, sothat the caption decoder 416 decodes caption data based on the captionservice information. Contrary to the embodiment of FIG. 11, in thepresent exemplary embodiment, 3D caption information for implementing a3D caption is not included in the caption service information carried bythe caption service descriptor.

The caption decoder 416 receives the closed caption data cc_data( ) fromthe video decoder 410 and accumulates the closed caption data paircarried by the “cc_data_(—)1” and “cc_data_(—)2” fields in packet units.In addition, the caption decoder 416 depacketizes the caption packetdata based on the caption service information from the PSI/PSIPprocessor 414 to restore service block data for each service.Afterwards, the caption decoder 416 decodes and interprets the serviceblock data. Meanwhile, the caption decoder 416 receives the captionservice information cc_disparity( ) from the video decoder 410 anddetects the base view information as well as the 3D caption parameter,i.e., the horizontal disparity value or the depth value. The captiondecoder 416 generates caption display information based on theinterpreted service block data and the 3D caption parameter to provideto the graphic engine 218.

The graphic engine 218 receives the caption display information from thecaption decoder 416 and generates caption bitmaps for the base view andthe extended view in OSD signal form based on the caption displayinformation. Here, the base view may be the left image and the extendedview may be a right image. First, the graphic engine 218 constructs acaption bitmap for the base view using the base view informationdetermined by the “base_view” field and the extracted closed captiondata, and outputs the caption bitmap for the base view in OSD signalform. Subsequently, the graphic engine 218 constructs a caption bitmapfor the extended view using the 3D caption parameter, and outputs thecaption bitmap for the extended view in OSD signal form. When the 3Dcaption parameter is the horizontal disparity value, the caption bitmapfor the extended view can be constructed by shifting the caption bitmapfor the base view by the horizontal disparity. Meanwhile, if the 3Dcaption parameter is the depth value, caption data corresponding to thebase view is displayed on the display by use of the caption depth value,which is especially useful when depth image is additionally provided inthe broadcast signal.

The first mixer 220 mixes the left image caption bitmap from the graphicengine 218 to the left image signal from the video decoder 410. Thesecond mixer 222 mixes the right image caption bitmap from the graphicengine 218 to the right image signal from the video decoder 410. Theformatter 224 formats the left and right images superimposed withrespective caption bitmaps while comparing frame time of the signals,such that temporally coincident left and right images can be displayedas a pair on a display panel 232. The formatted 3D image signal isreconstructed into a 3D image on the display panel 226.

Meanwhile, the image synthesis performed by the first and second mixers220 and 222 may be selectively carried out in response to a usercommand. That is, the user may manipulate, for example, a remotecontroller to choose whether a closed caption window is to be displayedor not and what closed caption window among a plurality of closedcaption windows is to be displayed. When such a manipulation command isapplied, a controller of the receiver may store the command in thememory and control the display of the closed caption windows withreference to the stored command.

The process of extracting closed caption information in the televisionreceiver illustrated in FIG. 27 will now be described in detail withreference to FIG. 28.

First, the PSI/PSIP processor 414 parses the PSIP to extract a MasterGuide Table (MGT) of which “PID” is ‘0x1FFB’ and “table_ID” is ‘0xC7’based on the packet ID (PID) and a table ID, to obtain PID informationof the Event Information Table (EIT) from the MGT. Next, the PSI/PSIPprocessor 414 reads a TP having the acquired PID to extract an EIT ofwhich “table_ID” is ‘0xC7’. Subsequently, the PSI/PSIP processor 414parses the EIT to detect the caption service descriptor shown in FIG. 26for each event in the EIT and the caption service information (step450).

In steps 452 to 466, the 3D caption parameter and the base viewinformation are extracted from the 3D caption informationdisparity_data( ) included in the picture header of the video ES andstored in the memory. First, in step 452, the video decoder 410 parsesthe video ES corresponding to the EIT to output extension and user dataand an MPEG-2 video bit stream, and decodes the MPEG-2 video bit stream.During the process of parsing the video ES, the video decoder 410decodes extension_and_user_data(2) in the picture header to extractpicture user data user_data( ) after extracting a header and theextension and user data. Subsequently, the video decoder 410 detectsATSC_user_data( ) having a “user_data_identifier” value of ‘0x4741 3934’from the picture user data user_data( ) (step 454). Thereafter, thevideo decoder 410 extracts user_data_type_structure( ) having“user_data_type_code” of ‘0x04’ from the ATSC_user_data( ) to provide tothe caption decoder 416 (456). Subsequently, the caption decoder 416extracts “cc_(—)3d_type”, “base_view”, and “cc_disparity” or “cc_depth”field values from the 3D caption information disparity_data( ) from thevideo decoder 410 (step 458).

Here, the “cc_(—)3d_type” field value determines whether to store the“cc_disparity” or “cc_depth” field value. If the “cc_(—)3d_type” fieldhas a value of ‘01’ or ‘10’ (step 460), the caption decoder 416 storesthe horizontal disparity “cc_disparity” value of the 3D captionparameter (step 462). In step 460, if the “cc_(—)3d_type” field has avalue of ‘11’, the caption decoder 416 stores the caption depth“cc_depth” value of the 3D caption parameter (step S464). On the otherhand, if the “_cc_(—)3d_type” field has a value of ‘00’, the captiondecoder 416 regards the caption service as a 2D caption service, anddiscards the “cc_(—)3d_type” field and the subsequent data by regardingthem as meaningless reserved bits (step 466).

Meanwhile, in an alternative embodiment where the video ES is encodedaccording to the H.265/AVC standard, the 3D caption information, i.e.,the disparity_data( ) can be also delivered to the receiver through theSEI Raw Byte Sequence Payload (RBSP) similarly that the closed captiondata cc_data( ) can be transferred through the SEI RBSP. In this case,the video decoder 410 parses an AVC NAL unit to extract SEI data havinga “nal_unit_type” value of ‘6’, and readsuser_data_registered_itu_t_t35( ) having a “payloadType” value of ‘4’.Thereafter, the video decoder 410 detects user_data( ) having the“user_data_identifier” value of ‘0x4741 3934’ from the picture user datauser_data( ). Subsequently, the video decoder 410 extracts auser_data_type_structure( ) having a “user_data_type_code” of ‘0x04’from the user_data( ). Then, the caption decoder 416 reads the 3Dcaption information disparity_data( ) structure from the extracted datato extract “cc_(—)3d_type”, “base_view”, and “cc_disparity” or“cc_depth” field values and store in the memory.

Meanwhile, in the process of parsing the video ES by the video decoder410, the cc_data( ) included in the picture header or a message of thevideo ES is extracted and provided to the caption decoder 416. Namely,the video decoder 410 detects the ATSC_user_data( ) having the“user_data_identifier” value of ‘0x4741 3934’ from the picture user datauser_data( ) of the video ES, and extracts the user_data_type_structure() having the ‘user_data_type_code” of ‘0x03’ from the ATSC_user_data( )as the closed caption data cc_data( ). The caption decoder 416accumulates the caption data byte pairs (cc_data1 and cc_data2) receivedthrough cc_data( ) in packet units, and extracts a service block withinthe packet packets to manage a caption service (step 468).

The process of displaying the extracted closed caption information onthe screen is similar to that described above with reference to FIGS. 13through 23. Namely, assuming that the caption bitmap for the base view,e.g., the left image, is similar to that shown in FIG. 14, the captionbitmap for the extended view shown in FIG. 16 may be constructed byshifting the caption bitmap for the base view by the horizontaldisparity as shown in FIG. 15. The first mixer 220 combines the leftcaption OSD signal to the left signal, and the second mixer 222 combinesthe right caption OSD signal to the right image signal. The formatter224 formats the two image signals according to the stereoscopic scheme,so that the formatted image signals are output through the display panel226. The left and right images superimposed with respective closedcaption windows may be synthesized according to the horizontal orvertical interleaving scheme as shown in FIG. 18 or FIG. 19,respectively, or may be synthesized into the checkerboard pattern asshown in FIG. 20.

FIG. 29 is a schematic block diagram of an alternative embodiment of thetelevision receiver of FIG. 27. According to the present embodiment, theleft and right broadcast image signals are formatted by one formatter,and left and right closed caption OSD signals are formatted by anotherformatter, and then the formatted signals are mixed. The first formatter224A formats the left and right image signals from the video decoder 410according to a stereoscopic output scheme. The second formatter 224Bformats the left and right image caption OSD signals from the graphicengine 218 according to the stereoscopic output scheme. The mixer 220Asynthesizes output signals of the first and second formatters 224A and224B, so that the formatted image signals are displayed on the displaypanel 226.

Assuming that the caption bitmaps for the left and right images are thesame as those shown in FIGS. 14 and 16, respectively, the secondformatter 224B outputs a caption bitmap image formatted in theside-by-side format shown in FIG. 22 or the top-down format shown inFIG. 23. Accordingly, the stereoscopic 3D image displayed on the displaypanel 226 after being mixed by the mixer 220A may have a form similar tothat shown in FIG. 18 or FIG. 19. Other features of the televisionreceiver illustrated in FIG. 29 are similar to those of the embodimentof FIG. 27, and thus a detailed description thereof will be omitted.

Third Embodiment

FIG. 30 illustrates a broadcasting system according to a third exemplaryembodiment of the present invention. In the third exemplary embodiment,3D caption information is inserted as a caption control code into theclose caption data cc_data( ).

In the system of FIG. 30, a video encoder 508 receives user data from auser data inserter 516 to insert into a picture header while encodingvideo signals. Particularly, the user data includes closed caption datacc_data( ), which then includes 3D caption information for supportingthe 3D caption.

A caption data generator 514 receives time-stamped caption data througha network or by a recording medium from an external workstation, andgenerates a caption channel packet shown in FIG. 3. The caption datagenerator 314 provides the caption channel packet and the 3D captioninformation to the user data inserter 316, so that the user datainserter 316 inserts these information into the picture user data. Inparticular, in the present exemplary embodiment, the 3D captioninformation is inserted as a control code into caption packet data ofthe caption channel packet as described below. In addition, the captiondata generator 314 provides caption service information indicating atype and attributes of a caption service to a PSI/PSIP generator 518 sothat the PSI/PSIP generator 518 can generate a caption servicedescriptor.

The user data inserter 516 provides the video encoder 508 with extensiondata and user data to be inserted at a sequence level, a GOP level, or apicture level during the encoding process. In particular, the user datainserter 516 provides a video encoder 508 with the caption channelpacket from the caption data generator 514 as a sort of user data, sothat the video encoder 508 can insert the caption channel packets by twobytes at a time into the picture header while encoding the video signal.

As mentioned above, when the caption data cc_data( ) is encodedaccording to a code space of FIG. 5 before a service block header and acaption header are added, the 3D caption information for supporting the3D caption is inserted into the caption data cc_data( ) through acontrol code. FIG. 31 illustrates an example of a coding format of a Set3D parameter (S3D) command for delivering the 3D caption information. Ina preferred exemplary embodiment, the Set 3D parameter command includesthree bytes: one byte command code and two bytes of parameters, and canbe abbreviated by “S3D” in the code space.

In the drawing, “3D_Type_Flag” field indicates whether or not aparameter for supporting the 3D caption is included, and a type of theparameter. The “3D_Type_Flag” value of ‘00’ indicates that the 3Dcaption is not supported and the parameters following this field shouldbe discarded. The “3D_Type_Flag” value of ‘01’ indicates that the 3Dcaption is supported and a “Disparity Parameter” field denotes ahorizontal disparity value expressed in the pixel units. The“3D_Type_Flag” value of ‘10’ indicates that the 3D caption is supportedand the “Disparity Parameter” field denotes a horizontal disparity valuerepresented as a ratio (%) with respect to the width of a display. The“3D_Type_Flag” value of ‘11’ indicates that the 3D caption is supportedand the “Disparity Parameter” field is a caption depth value.

“Base_View” field defines a base view which serves as a reference of thestereoscopic image pair, i.e., the left and right images. In the presentexemplary embodiment, the left view is the base view if the “base_view”field is 0, and the right view is the base view if the “base_view” fieldis 1.

When the “3D_Type_Flag” field has a value of ‘01’ or ‘10’, the“Disparity Parameter” field is a horizontal disparity value between thebase view and the extended view at a caption display position, anddenotes a caption display position for the extended view with respect tothe caption display position for the base view. Thus, the receiver canobtain the closed caption display position for the extended view byshifting the closed caption for the base view by the horizontaldisparity value. As mentioned above, the data format of the “DisparityParameter” field may be indicated by the “3D_Type_Flag” field. In theabove description, the horizontal disparity value denoted by the“3D_Type_Flag” field can be expressed by the number of pixels or a ratio(%) with respect to the width of the display. Alternatively, however,the “3D_Type_Flag” may be expressed by physical length (e.g.,centimeters, inches, etc.) as well. When the horizontal disparity valueis intended to be expressed by the physical length, the “3D_Type_Flag”field may be defined to substitute either the pixel numbers or the ratioto the display width, or a certain field may be newly defined besidesthe “3D_Type_Flag” field. In addition, the “3D_Type_Flag” field value of‘00’ may define that the “Disparity Parameter” denotes a horizontaldisparity value expressed by the pixel numbers.

When the “3D_Type_Flag” field has a value of ‘11’, the “DisparityParameter” field denotes the depth of the caption in the stereoscopic 3Dimage. When the 3D video data includes depth information, the depth datafor the caption data can be provided to the receiver instead of thehorizontal disparity, so that the receiver can determine the captiondisplay position in the 3D images by applying the depth data.Preferably, the size and scale of the depth is equal to depthinformation regarding video data so that the receiver can interpret itin the same manner. Even though the depth value for the video data mayhave a negative or positive value, a depth map which maps the depthvalue into a value of [0,255] may be constructed. Preferably, the depthvalue for the caption is expressed in this manner. Meanwhile, inconstructing the depth map, the depth value may be mapped into the valueof [0,255] in a linear scale. Alternatively, however, the depth valuemay also be mapped, for example, in a logarithmic scale in considerationof the perception characteristics of human beings.

FIG. 32 illustrates the position of the Set 3D parameter (S3D) commandin a code space according to an exemplary embodiment of the presentinvention. In the code space compliant with the CEA-708-B standard ofFIG. 5, C0 code set is allocated to map various control codes, andincludes 32 codes from “0x00” to “0x1F.” In the C0 code set, the codes“0x00” through “0x0F” are single byte codes, and the codes “0x10”through “0x17” are two byte codes, and the codes “0x18” through “0x1F”are three byte codes.

In the embodiment described above, the Set 3D parameter command is athree byte code, and can be mapped into one of empty codes among thecodes “0x18” through “0x1F.” Preferably, the Set 3D parameter command ismapped into the position of “0x18.”

The Set 3D parameter command defined as above constitutes service blockdata along with another control codes and caption text data. The serviceblock data prefixed by a service block header forms a service block asshown in FIG. 4, and the service block prefixed by a packet headerconstitutes a caption channel packet as shown in FIG. 3. The captionchannel packet is divided in two byte units and inserted into thepicture header of the video ES.

Caption service information describing the type and attributes of acaption service carried by the picture user data is transmitted to thereceiver separately from the caption data. In the present exemplaryembodiment, the caption service information is generated according tothe ATSC A/65 standard, included in a caption service descriptor, andtransmitted through an Event Information Table (EIT) of an MPEG-2transport stream. The syntax of the caption service descriptor accordingto the present exemplary embodiment follows the general format shown inFIG. 26.

FIG. 33 is a schematic block diagram of a television receiver suitablefor receiving the broadcast signal transmitted by the system of FIG. 30.

The tuner 200 selects a broadcast signal of a certain channel selectedby the user among a plurality of broadcast signals inputted through anantenna. The demodulation and channel decoding unit 202 demodulates thebroadcast signal from the tuner 200, and performserror-correction-decoding on the demodulated signal to output theTransport Stream (TS). The transport demutiplexer 204 demultiplexes theTS to separate a video ES and an audio ES and extract PSI/PSIPinformation.

The audio decoder 208 decodes the audio ES to output a digital audio bitstream. The audio bit stream is converted into an analog audio signal bythe digital-to-analog converter, amplified by the amplifier, and thenoutputted through the speaker. A video decoder 610 parses the video ESto extract extension and user data and MPEG-2 video bit stream. Thevideo decoder 610 extracts closed caption data cc_data( ) from theextension and user data and to output to the caption decoder 216. Thevideo decoder 610 decodes the MPEG-2 video bit stream to output left andright video signals for implementing stereoscopic 3D image pair. Thedecoding process performed by the audio decoder 208 and the videodecoder 610 may be carried out based on the packet ID (PID) detected bythe PSI/PSIP processor 214.

The PSI/PSIP processor 414 receives and parses the PSI/PSIP informationfrom the transport demultiplexer 204 to store in the memory or theregister, so that the broadcast images are reproduced based on thestored information. The EIT of the PSIP includes a caption servicedescriptor which contains the caption service information describing thetype and attributes of a caption service. The PSI/PSIP processor 214provides the caption service information to a caption decoder 616, sothat the caption decoder 616 can decode caption data based on thereceived caption service information.

The caption decoder 616 receives the closed caption data cc_data( ) fromthe video decoder 610 and accumulates the closed caption data pairscarried by the “cc_data_(—)1” and “cc_data_(—)2” fields. In addition,the caption decoder 616 depacketizes the caption packet data based onthe caption service information from the PSI/PSIP processor 414 torestore service block data for each service. And then, the captiondecoder 616 decodes and interprets the service block data. When the Set3D parameter command is detected in the process of interpreting theservice block data, the caption decoder 616 detects base viewinformation as well as the 3D caption information parameter, i.e., thehorizontal disparity value or the depth value from the command. Thecaption decoder 616 generates caption display information based on theinterpreted service block data and the 3D caption parameter to providethe graphic engine 218.

The graphic engine 218 receives the caption display information from thecaption decoder 416 and generates caption bitmaps for the base view andthe extended view in OSD signal form based on the caption displayinformation. Here, the base view may be the left image and the extendedview may be the right image. First, the graphic engine 218 constructsthe caption bitmap for the base view using the base view informationdetermined by the “Base_View” field and the extracted closed captiondata, and outputs the caption bitmap for the left image in the OSDsignal form. Subsequently, the graphic engine 218 constructs the captionbitmap for the extended view using the 3D caption parameter, and outputsthe caption bitmap for the right image in the OSD signal form. When the3D caption parameter is the horizontal disparity value, the captionbitmap for the extended view can be constructed by shifting the captionbitmap for the base view by the horizontal disparity. Meanwhile, if the3D caption parameter is the depth value, caption data corresponding tothe base view is displayed on the display by use of the caption depthvalue, which is especially useful when depth image is additionallyprovided in the broadcast signal.

The first mixer 220 mixes the left image caption bitmap from the graphicengine 218 to the left image signal from the video decoder 610. Thesecond mixer 222 mixes the right image caption bitmap from the graphicengine 218 to the right image signal from the video decoder 610. Theformatter 224 formats the left and right images superimposed withrespective caption bitmaps while comparing a frame time of the imagesignals, such that temporarily coincident left and right images can bedisplayed as a pair on the display panel 232. The formatted 3D imagesignal is reconstructed into a 3D image on the display panel 226.

The process of extracting closed caption information in the televisionreceiver illustrated in FIG. 33 will now be described in detail withreference to FIG. 34. First, the PSI/PSIP processor 414 parses the PSIPto extract a Master Guide Table (MGT) of which “PID” is ‘0x1FFB’ and“table_ID” is ‘0xC7’ based on a packet ID (PID) and a table ID, andobtains PID information regarding an Event Information Table (EIT) fromthe MGT. Next, the PSI/PSIP processor 414 reads a TP having the acquiredPID to extract an EIT of which “table_ID” is ‘0xC7’. And then, thePSI/PSIP processor 414 parses the EIT and detects the caption servicedescriptor shown in FIG. 26 for each event in the EIT as well as captionservice information (step 650).

In step 652, the video decoder 610 parses the video ES corresponding tothe EIT to output extension and user data and an MPEG-2 video bitstream, and decodes the MPEG-2 video bit stream. During the process ofparsing the video ES, the video decoder 610 decodesextension_and_user_data(2) in the picture header to extract picture userdata user_data( ) after extracting the header and the extension and userdata. Meanwhile, in an alternative embodiment where the video ES isencoded according to the H.264/AVC standard, a Supplemental EnhancementInformation (SEI) is decoded at this stage. Namely, the video decoder610 parses an AVC NAL unit to extract the SEI data having an“nal_unit_type” value of ‘6’ and reads user_data_registered_itu_t_t35( )message having “payloadType” value of ‘4’.

Subsequently, the video decoder 610 detects ATSC_user_data( ) having“user_data_identifier” value of ‘0x4741 3934’ from the picture user datauser_data( ), and extracts user_data_type_structure( ) having“user_data_type_code” of ‘0x03’ from the ATSC_user_data( ) as the closedcaption data cc_data( ). Subsequently, the caption decoder 616accumulates the caption data byte pairs (cc_data1 and cc_data2) receivedthrough cc_data( ) in packet units, and extracts a service block withinthe packet packets to manage a caption service (step 654).

Thereafter, the caption decoder 616 detects a data byte corresponding tothe Set 3D parameter command from the accumulated caption service blockdata (step 656). The caption decoder 616 extracts “3D_Type_Flag”,“Base_View”, “3D_Type_Flag”, and “Disparity/Depth Parameter” fields fromtwo parameter bytes following the detected command byte (step 658).

In step 660, the caption decoder 616 determines whether a close captionshould be displayed in 3D based on the “3D_Type_Flag” field. If the“3D_Type_Flag” field has a value of ‘01’, ‘10’, or ‘11’ to indicate the3D caption, the caption decoder 616 extracts a 3D caption-relatedparameters from the Set 3D parameter command and stores it in thememory. In more detail, when the “3D_Type_Flag” field has a value of‘01’ or ‘10’, the caption decoder 616 stores the “Disparity/DepthParameter” as a horizontal disparity value (step 662). If the“3D_Type_Flag” field has a value of ‘11’, the caption decoder 616 storesthe “Disparity/Depth Parameter” as a caption depth value (step 664).Meanwhile, if the “3D_Type_Flag” field has a value of ‘00’, the captiondecoder 616 regards that the corresponding program does not support the3D caption function, and discards the parameter data within the Set 3Dparameter command (step 666).

The process of displaying the extracted closed caption information onthe screen is similar to that described above with reference to FIGS. 13through 23. FIG. 35 is a flow chart illustrating an example of theprocess of displaying the 3D closed caption in the present embodiment.Here, it is assumed that the 3D parameter is the horizontal disparityand the disparity is set to be optimized for a display panel havingresolution of 1920 pixels wide.

First, horizontal disparity value is read out of the memory andhorizontal size (referred to as “video width” hereinbelow) and aspectratio of broadcast images are checked from the video ES (step 700). Instep 702, the horizontal display value is scaled to be adapted for thedisplay panel and horizontal position of the caption window isdetermined based on the horizontal disparity value and the video widthinformation.

Referring to FIG. 36, an actual horizontal display value for using inrendering of the caption images is determined by multiplying a ratio ofthe video width to 1920 by the received horizontal disparity value. Forexample, when the received horizontal disparity value is 24 and thevideo width is 1920, horizontal disparity value of 24 is used forrendering of the caption images. If the received horizontal disparityvalue is 24 and the video width is 1280, horizontal disparity value of16 (=24*1280/1920) is used for rendering of the caption images. If thereceived horizontal disparity value is 24 and the video width is 720,horizontal disparity value of 9 (=24*720/1920) is used for rendering ofthe caption images.

In case that the aspect ratio is 16:9, a position (specifically, anchorposition) of the caption window for the left image is determined bymultiplying a ratio of the received anchor position data to 210 by thevideo width, where the numeral ‘210’ corresponds to cell numbers inhorizontal direction for a 16:9 display according to the CEA-708-Bstandard. If the aspect ratio is 4:3, the anchor position of the captionwindow for the left image is determined by multiplying a ratio of thereceived anchor position data to 160 by the video width, where thenumeral ‘160’ corresponds to cell numbers in horizontal direction for a4:3 display according to the CEA-708-B standard.

Subsequently, an anchor position of the caption window for the rightimage is determined by adding the scaled horizontal disparity value tothe anchor position of the caption window for the left image (step 704).

Afterwards, the caption bitmap for the left image is constructedaccording to received caption text and control command codes. Also, thecaption bitmap for the right image is constructed by shifting thecaption bitmap for the left image by the amount of the scaled horizontaldisparity value (step 706).

In step 708, the left and right caption bitmaps are mixed with the leftand right broadcast images, respectively. At this stage, since theresolution of the 3D broadcast images are half of that of the captionbit maps, the caption bit maps are sampled by a rate of ½ according tothe format of the 3D broadcast images so that the resolution of thesignals become to be the same as each other. That is, the caption bitmaps are sampled line-by-line if the 3D broadcast images are formattedin top-down form, while being sampled column-by-column if the 3Dbroadcast images are formatted in side-by-side form.

Finally, the left and right broadcast images mixed with caption bitmapsare formatted, so that 3D broadcast images are output through thedisplay panel 226 along with the 3D caption (step 710).

FIG. 37 illustrates an exemplary command sequence for displaying the 3Dcaption in the television receiver of FIG. 33. First, Define Window(DFx) command may be presented to create a caption window and setinitial window parameters such as window size and position. Next, SetCurrent Window (CWx) command may be presented to specify current window.Subsequently, Set Window Attributes (SWA) command may be presented todefine the window styles for the current window such as justification,print direction, and fill color. Afterwards, Set Pen Attributes (SPA),Set Pen Color (SPC), and Set Pen Location (SPL) commands may bepresented to define font style, pen color, and pen cursor location,respectively. The Set 3D parameter (Set 3D parameter: S3D) command maybe presented anytime after the current window is set by the CWx command.In a state that the 3D parameter delivered by the S3D command is storedin the memory, the television receiver can receive the caption text dataand display the 3D caption text using the 3D parameter.

FIG. 38 is a schematic block diagram of an alternative embodiment of thetelevision receiver of FIG. 33. According to the present embodiment, theleft and right broadcast image signals are formatted by one formatter,and left and right closed caption OSD signals are formatted by anotherformatter, and then the formatted signals are mixed. The first formatter224A formats the left and right image signals from the video decoder 410according to a stereoscopic output scheme. The second formatter 224Bformats the left and right image caption OSD signals from the graphicengine 218 according to the stereoscopic output scheme. A mixer 220Asynthesizes output signals of the first and second formatters 224A and224B, so that the formatted image signals are displayed on the displaypanel 226.

Assuming that the caption bitmaps for the left and right images are thesame as those shown in FIGS. 14 and 16, respectively, the secondformatter 224B outputs a caption bitmap image formatted in theside-by-side format shown in FIG. 22 or the top-down format shown inFIG. 23. Accordingly, the stereoscopic 3D image displayed on the displaypanel 226 after being mixed by the mixer 220A may have a form similar tothat shown in FIG. 18 or FIG. 19. Other features of the televisionreceiver illustrated in FIG. 38 are similar to those of the embodimentof FIG. 33, and thus a detailed description thereof will be omitted.

Even though the “Disparity Parameter” field consists of one byte dataaccording to the coding format of FIG. 31, the Disparity Parameter maycomprise two or more bytes alternatively so as to enhance precision ofdata or extend display range. FIG. 39 shows such an example. In theexample of FIG. 39, the Set 3D parameter command includes four bytes:one byte command code and three bytes of parameters. Here, the“Disparity Parameter” field has two bytes and is expressed by one upperbyte and one lower byte.

FIG. 40 illustrates the position of the Set 3D parameter command in thecode space according to an exemplary embodiment of FIG. 39. In the codespace according to the CEA-708-B standard shown in FIG. 5, captioncontrol codes are concentrated in the C1 code set. If an additionalcommand is required besides the code set, reserved codes in C2 or C3code set are available. C2 code set includes 32 codes from “0x00” to“0x1F”, and may be addressed by prefixing “0x10” (i.e., ‘EXT1’ code).Among the codes in the C2 code set, codes from “0x00” to “0x07” aresingle byte codes (requiring no additional byte), codes from “0x08” to“0x0F” are two byte codes (requiring one additional byte), codes from“0x10” to “0x17” are three byte codes (requiring two additional bytes),and codes from “0x18” to “0x1F” are four byte codes (requiring threeadditional bytes). In the exemplary embodiment of FIG. 39, the Set 3Dparameter (S3D) command consists of four bytes excluding the prefix, the‘EXT1’, and thus can be mapped into one of the codes from “0x18” to“0x1F.” Preferably, the Set 3D parameter (S3D) command is mapped intothe code “0x18.”

On the other hand, the Set 3D parameter (S3D) command defined in FIGS.31 and 39 is presented after the caption window for applying the 3Dparameter is activated by Set Current Window (CWx) command. Here, theterm “activation” refers to a condition that the caption window is setto the current window so that additional caption text can be revealed inthe caption window. However, unless the 3D parameter is not delivered tothe receiver in advance, a delay may happen before the 3D caption datais processed and the 3D caption is displayed in the receiver.Furthermore, in case that another caption window is set as the currentwindow and the original caption window just maintains the previouscaption text without displaying additional caption text, it is possiblethat the 3D parameter for the original caption window is replaced bythat of the new window so that 3D parameter for the original captionwindow is not maintained in the memory. If a plurality of windows whichinclude caption data exists in a screen, 3D parameter has to be definedto each window in advance.

An example that two caption services are alternately provided to thereceiver is described with reference to FIG. 41. caption window 1 may becreated by Define Window (DF1) command, the caption window 1 may be setas a current window by Set Current Window (CW1) command, and windowstyles for the caption window 1 may be defined by Set Window Attributes(SWA) command. Afterwards, pen styles for the caption window 1 may bedefined by Set Pen Attributes (SPA), Set Pen Color (SPC), or Set PenLocation (SPL) command, and caption text may be received to display the3D caption in the caption window 1 after the Set 3D parameter (S3D)command is delivered. In this state, caption window 2 may be created byDefine Window (DF2) command, the caption window 2 may be set as thecurrent window by Set Current Window (CW2) command, and window stylesfor the caption window 2 may be defined by Set Window Attributes (SWA)command. Afterwards, pen styles for the caption window 2 may be definedby Set Pen Attributes (SPA), Set Pen Color (SPC), or Set Pen Location(SPL) command, and caption text may be received to display the 3Dcaption in the caption window 2 after the Set 3D parameter (S3D) commandis delivered.

If the caption window 1 is to be activated again in this state, thecaption window 1 must be set as the current window by Set Current Window(CW1) command, and the 3D parameter for the caption window 1 probablyhas to be delivered again by the Set 3D parameter (S3D) command beforereceiving the caption text. Similarly, if the caption window 2 is to beactivated again in this state, the caption window 2 must be set as thecurrent window by Set Current Window (CW2) command, and the 3D parameterfor the caption window 1 probably has to be delivered again by the Set3D parameter (S3D) command before receiving the caption text.

Thus, in order to reduce the necessity of frequent delivery of the 3Dparameter and prevent the delays, it is desirable to allow Set 3Dparameter (S3D) command be provided to the receiver for each captionwindow separately disregarding what caption window is set for thecurrent window, so that the 3D parameter is set and maintainedindependently from the current window. Also, the 3D parameter isrepetitively delivered to the receiver regardless of the activatedcaption window, periodically or aperiodically (e.g., when a substantialcaption data rate is low).

FIG. 42 illustrates the coding format of the Set 3D parameter (S3D)command suitable for delivering the 3D parameter regardless of theactivated caption window. In the present embodiment, the Set 3Dparameter (S3D) command includes three bytes: one byte command code andtwo bytes of parameters. The Set 3D parameter is defined for eachwindow, and thus a plurality of 3D parameters can be deliveredsequentially for the windows. For example, if a plurality of windows,each window including different caption data, exists in a screen, the 3Dparameter for each window has to be delivered separately.

In the drawing, bits id3-id0 indicates the ID of a caption window forapplying the 3D parameter. A bit dp_sign indicates the sign of the 3Dparameter, and bits dp10-dp0 denotes the horizontal disparity betweenthe caption bit map images for the left and right images. Since thehorizontal disparity is expressed in eleven bits, the horizontaldisparity value for a display panel having resolution of 1920 pixelswide can be represented sufficiently.

FIG. 43 illustrates the position of the set 3D parameter (S3D) commandof FIG. 42 in the code space. Since the Set 3D parameter (S3D) commandof FIG. 40 is a three byte control code, the code can be mapped into oneof empty codes among the codes “0x18” through “0x1F”. Preferably, theSet 3D parameter (S3D) command is mapped into the position of “0x11” inC2 code set. Alternatively, however, the command can be mapped into oneof the codes in the C0 code set, for example, “0x19.”

The Set 3D parameter command defined as above constitutes service blockdata along with another control codes and caption text data. The serviceblock data prefixed by a service block header forms a service block asshown in FIG. 4, and the service block prefixed by a packet headerconstitutes a caption channel packet header as shown in FIG. 3. Thecaption channel packet is divided in two byte units and inserted intothe picture header of the video ES.

Meanwhile, any receiver incapable of displaying the 3D caption accordingto the present invention recognizes the Set 3D parameter command asundefined byte codes and ignores the codes.

FIG. 44 illustrates an example of using the Set 3D parameter (S3D)command of FIG. 42 along with other caption control commands in thetelevision receiver of FIG. 33. As shown in the drawing, the Set 3Dparameter (S3D) command for the caption window 1 can be deliveredanytime even before the caption window 1 is activated by the Set CurrentWindow (CW1) command, as long as the caption window 1 is defined. Also,the Set 3D parameter (S3D) command for the caption window 2 can bedelivered anytime even before the caption window 2 is activated by theSet Current Window (CW2) command, as long as the caption window 2 isdefined. Thus, the 3D caption can be displayed stably without any timedelay. Also, the Set 3D parameter (S3D) command can be deliveredperiodically or aperiodically regardless of the activated caption windowto refresh the 3D caption parameter.

Other Alternatives

Besides, various changes and modifications may be made to the describedembodiments without departing from the spirit and scope of the presentinvention.

For example, even though the caption service descriptor is included inthe Event Information Table (EIT) in the exemplary embodiments describedabove, the descriptor may be inserted into a Program Mapping Table (PMT)or the other table, alternatively.

Regardless of the illustrative description for each embodiment above,either the horizontal disparity value or the depth value may be includedin the 3D caption information, and both data may be included as well.Besides, the flag or field indicating whether the caption is threedimensional is optional, and may not be included in the 3D captioninformation. Furthermore, the flag or field designating the base view isoptional also, and can be omitted. In such a case, the base view can bedesignated by default (e.g., as the left image) or in the video ESlevel.

The name of the Set 3D parameter (S3D) command and its abbreviation canbe designated to be “SetDepthPos” or the other one arbitrarily.

Meanwhile, in the above description, the 3D caption parameter, e.g., thehorizontal disparity value or the caption depth value, is transmitted tothe receiver through broadcast signal. However, the horizontal disparityvalue may be synthesized in the receiver. For example, the televisionreceiver may apply a block matching algorithm for the left and rightimages to calculate horizontal disparity values with respect to aplurality of blocks or objects. Generally, an object having a positivehorizontal disparity of larger absolute value is recognized to bepositioned farther from the viewer, and another object having a negativehorizontal disparity of larger absolute value is recognized to bepositioned nearer to the viewer. Therefore, a horizontal disparity valuefor the closed caption can be determined based on a horizontal disparityvalue calculated for an object presumed to be positioned nearest to theviewer.

If the screen has a large size and the horizontal disparity valuebetween the closed captions for the left and right images is large,there may be a difference between the width of the closed caption forthe left image and the width of the closed caption for the right image.Thus, in consideration of this, the graphic engine may additionallyperform scaling in order to adjust the width of one or both of theclosed captions for the left and right images.

Thus, it should be understood that the forgoing description isillustrative and nor restrictive. We claim all modifications andvariation coming within the spirit and scope of the following claims.

We claim:
 1. A method of controlling a broadcast receiving apparatus,the method comprising: parsing a video elementary stream to extractpicture user data and a video bit stream; extracting closed caption datacomprising a 3D parameter command from the picture user data, whereinthe closed caption data is encoded using a code space that contains basecode sets and extended code sets; generating at least one of a base viewcaption window and an extended caption window using the closed captiondata and the 3D parameter command; and displaying a 3D caption using thebase view caption window and the extended caption window with a 3D imagesupported by decoding the video bit stream, wherein the broadcastreceiving apparatus is configured to support an output of not only the3D caption but also a 2D caption, and wherein a determination ofdisplaying whether the 2D caption or the 3D caption is based on a user'sselection of a specific caption output.
 2. The method of claim 1,wherein a control command reflecting the user's selection is receivedfrom an external remote controller.
 3. The method of claim 2, wherein ifthe control command is for displaying the 2D caption, the 3D caption ischanged into the 2D caption, and wherein the 2D caption includes onlyone of the base view caption window and the extended caption window. 4.The method of claim 3, wherein the 2D caption is displayed with the 3Dimage when the 3D caption is changed into the 2D caption.
 5. The methodof claim 1, wherein information related to the determination ofdisplaying whether the 2D caption or the 3D caption is stored such thatthe information is used to determine to display whether the 2D captionor the 3D caption in next time.
 6. The method of claim 1, wherein aplurality of 3D captions are simultaneously displayed with the 3D image,and wherein locations of the plurality of 3D captions are different fromeach other.
 7. The method of claim 6, wherein the 3D parameter commandcomprises one or more disparity values for displaying the plurality of3D captions, and wherein each the disparity information for displayingthe plurality of the 3D caption corresponds to each pair of the baseview caption window and the extended caption window.
 8. The method ofclaim 1, wherein the 3D parameter command comprises one or moredisparity values for displaying the 3D caption, and wherein the extendedcaption window is generated by applying the disparity values to the baseview caption window.
 9. The method of claim 8, wherein the disparityvalues are specified using a horizontal resolution of 1920 units. 10.The method of claim 9, wherein for signals delivered at a resolution of1920 pixels horizontally, one unit of disparity corresponds to onepixel, and wherein for signals delivered at a resolution of 1280 pixelshorizontally, one unit of disparity corresponds to 1920/1280 pixels. 11.The method of claim 1, wherein the 3D parameter command is delivered inan extended code set, and wherein the extended code set is accessed byusing an ‘EXT1’ code in a base code set.
 12. A broadcast receivingapparatus, comprising: a demultiplexing unit; a decoding unit; aprocessor unit; and a display unit configured to display a 3D image,wherein the demultiplexing unit, the decoding unit, the processor unitand the display unit cooperate with each other to: parse a videoelementary stream to extract picture user data and a video bit stream;extract closed caption data comprising a 3D parameter command from thepicture user data, wherein the closed caption data is encoded using acode space that contains base code sets and extended code sets; generateat least one of a base view caption window and an extended captionwindow using the closed caption data and the 3D parameter command; anddisplay a 3D caption using the base view caption window and the extendedcaption window with the 3D image supported by decoding the video bitstream on the display, wherein the broadcast receiving apparatus isconfigured to support an output of not only the 3D caption but also a 2Dcaption, and wherein a determination of displaying whether the 2Dcaption or the 3D caption is based on a user's selection of a specificcaption output.
 13. The apparatus of claim 12, wherein a control commandreflecting the user's selection is received from an external remotecontroller.
 14. The apparatus of claim 13, wherein if the controlcommand is for displaying the 2D caption, the 3D caption is changed intothe 2D caption, and wherein the 2D caption includes only one of the baseview caption window and the extended caption window.
 15. The apparatusof claim 14, wherein the 2D caption is displayed with the 3D image whenthe 3D caption is changed into the 2D caption.
 16. The apparatus ofclaim 12, wherein information related to the determination of displayingwhether the 2D caption or the 3D caption is stored such that theinformation is used to determine to display whether the 2D caption orthe 3D caption in next time.
 17. The apparatus of claim 12, wherein aplurality of 3D captions are simultaneously displayed with the 3D image,and wherein locations of the plurality of 3D captions are different fromeach other.
 18. The apparatus of claim 17, wherein the 3D parametercommand comprises one or more disparity values for displaying theplurality of 3D captions, and wherein each the disparity information fordisplaying the plurality of the 3D caption corresponds to each pair ofthe base view caption window and the extended caption window.
 19. Theapparatus of claim 12, wherein the 3D parameter command comprises one ormore disparity values for displaying the 3D caption, and wherein theextended caption window is generated by applying the disparity values tothe base view caption window.
 20. The apparatus of claim 19, wherein thedisparity values are specified using a horizontal resolution of 1920units.